The present invention relates to an image pickup apparatus or an image sensing apparatus which uses a solid-state image pickup device having a widened the dynamic range.
It has been conventionally desired to broaden the dynamic range of a solid-state image pickup device, and various technique have been proposed for that purpose.
For example, Jpn. Pat. Appln. KOKAI Publication No. 5-308573 filed by the present inventor discloses a technique associated with an image pickup apparatus using an X-Y address type solid-state image pickup device (CMD: Charge Modulation Device) in which the integration time is controlled to read out two frames with different exposure amounts, and a correction means for correcting an image is used to widen the dynamic range.
Jpn. Pat. Appln. KOKAI Publication No. 7-38815 filed by the present inventor discloses a technique associated with a CMD, in which a read is performed twice during the horizontal period, thereby independently obtaining two pixel signals with different exposure times in one frame period.
FIG. 11 is a view showing the concept of the CMD.
Referring to FIG. 11, reference numeral 51 denotes a vertical scanning circuit; 52, a horizontal scanning circuit, and 53, a light-receiving unit in which photoelectric conversion devices are two-dimensionally arrayed as pixels.
The operation of the vertical scanning circuit 51 will be described with reference to the timing chart in FIG. 12. The vertical scanning circuit 51 sequentially applies a read voltage VRD to the first to nth lines within the vertical effective period of an image. Next, the horizontal scanning circuit 52 sequentially outputs a selection pulse, and a signal is sequentially read out from a pixel of a line applied with this voltage. A reset voltage VRS is applied to the line from which one line signal is completely read out. The charges accumulated in the pixel of the line applied with the reset voltage are reset, and the pixel is exposed until the next read period. This operation is repeated for the first to nth lines, thereby reading out one frame signal. Note that VOF represents an overflow voltage. By applying the overflow voltage VOF during the horizontal blanking period, unwanted holes for the signal are cleared.
In the X-Y address type solid-state image pickup device, signals are read out in this way, and the exposure timing shifts in units of lines, as shown in FIG. 13, on the basis of the vertical focal plane scheme.
However, for the image pickup apparatus disclosed in Jpn. Pat. Appln. KOKAI Publication No. 5-308573, the operation in use of a light source such as an electronic flash unit is not described.
As shown in FIG. 14, an electronic flash tube 20 has light-emitting characteristics in which the light amount reaches its maximum value at the initial stage of rising immediately after light emission and then attenuates. The light emission time is short relative to the video rate (a period when the image pickup device scans an image formed on the surface of the image pickup device and outputs one frame signal as a video signal). For this reason, when a short-time light-emitting means such as an electronic flash unit is used, the region actually exposed is not certain, and one frame may be only partially exposed.
In the image pickup apparatus described in Jpn. Pat. Appln. KOKAI Publication No. 5-308573, an image having a wider dynamic range is synthesized from two frames with different exposure amounts by an image correction means, and the integration time is controlled to obtain the two frames with different exposure amounts. When the exposure amount is proportional to the integration time, no problem is posed. However, when s short-time light-emitting means such as an electronic flash unit is used, the exposure amount is not proportional to the exposure time. For this reason, the image is not appropriately corrected by the image correction means.
In the image pickup device described in Jpn. Pat. Appln. KOKAI Publication No. 7-38815 as well, the exposure period is controlled to obtain frames with different exposure amounts. In this case as well, when the exposure amount is proportional to the integration time, no problem is posed. However, when a short-time light-emitting means such as an electronic flash unit is used, the exposure amount is not proportional to the exposure time. For this reason, images with different exposure amounts cannot be appropriately obtained.
The means for synthesizing an image with a wider dynamic range from two frames with different exposure amounts performs synthesis using the ratio of exposure amounts between the two frames. When the ratio of exposure amounts has a predetermined value, image synthesis is properly performed (the specific synthesizing method is described in detail in the above prior art). In the normal operation, images with different exposure amounts are obtained by controlling the exposure period. With this method, the ratio of exposure amounts is kept constant. However, the light-emitting means such as an electronic flash unit has variations in light emission amount due to variations in charge voltage or the like. For this reason, errors are generated in image synthesis by the image synthesizing means, resulting in a degradation in image quality.
The above-described Jpn. Pat. Appln. KOKAI Publication No. 5-308573 filed by the present inventor discloses an image pickup apparatus using an X-Y address type solid-state image pickup device, in which the integration time, i.e., the exposure time for accumulating charges is controlled to read out two images with different exposure amounts, and these two images are synthesized to broaden the dynamic range.
In the image pickup apparatus described in Jpn. Pat. Appln. KOKAI Publication No. 5-308573, the exposure time is controlled to obtain two images with different exposure amounts as synthesis targets. However, no consideration is given to measures which may be come necessary in use of a light source such as an electronic flash unit.
That is, the image pickup apparatuses described in the prior arts only control the exposure time to obtain two images with different exposure amounts. Such a technique poses no problem when the exposure amount is proportional to the exposure time in, e.g., an image pickup operation based on a light source such as natural light or an illumination lamp. However, when a flash for emitting light for a short time, e.g., an electronic flash unit is used as a light source, the exposure amounts of two images are not proportional to the exposure time, and the exposure amount of the short-time light emission predominantly determines the total light amount. Even when the picked up images are synthesized simply with reference to the exposure time, the images are not appropriately synthesized for the purpose of widening the dynamic range by covering the corresponding luminance level ranges by images with different exposure amounts.
In addition, if the object moves in the composition during the light-emitting interval, the two images shift from each other upon image synthesis processing.